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ABSTRACT ) 1 - _ 

, The more commonly cited uses of state and local 
district assessment .prog rams are addressed. The implications of seven 
proposed uses of state assessment data are reviewed point-by- point - 
(1) allocating state grants-in-iid to alleviate weaknesses, in * ^ 

instructional programs; (2) 'designing instructional support programs 
for tethers ; (3) developing state planning statements and 

priorities; (4) revising state minimum standards for schools; (5) 
reporting and making recommendations to the legislature; (6) 
determining if students are acquiring "survival level" skills or ’ 
"minimum competencies;" and (7) (determining the extent to which 
students in a state have attained the skills,- knowledge, and 
attitudes reflected in the educational goals of that state-. Next, the 
authbr comments on some uses of large-scale testing in school 
districts and some conditions that should be met if su^h uses ajjjS^to* 
be realized. Finally, several major obstacles inherent in developing ✓ 
measurement instruments and procedures in afeas other than reading,]/ 
language, and math are discussed. (RC) * , ]/ 



* *** ***** #*** 

* 

* 

* 

* 

* 

* 



Documen 
materials n 
to obtain X 
reprodu^ibi 
of the micr 
via the -SRI 
responsible 

♦ supplied by 

* *** ** 3f: 3?C3?C £ 



♦ *** **************** ************************* ************* 
ts acquired by ERIC include many informal, unpublished ♦ 
ot available from other sources. ERFC makes every effort ♦ 
he best copy available. Nevertheless, items of marginal * 
lity are often encountered and this affects the quality 
ofiche and haraicopy reproductions ERIC makes available 
C Document fe^bduction Service (EDPS) . EDRS is not 
for the guality of the original document. Reproductions 
EDRS are the best that can 'he made from the original. 
******************** ************************************** 



IU 



US DEPARTMENT Of HEALTH 
'EDUCATION 4 WELFARE 
national INSTITUTE OP 
EDUCATION 



\ 



Tm ’ s occ^v^nt „ AS 0geN OEPft0 
&UCED as SECEDED fQOV 

*Th£ PERSON Lft 3 »:,a\ J A * On Oft tG 1 n* 
AT lNG it PC % * ^ *> » EAOPOPiN ONS 
s T a ted r*c \ - Ne:essAe\. v cgpoe 
SENT C * ► f A NA'CNA. NS * T U T EOP 

EOo r «' jN PC ' ON CP PO^ C y 



Public School Perspectives On the Uses 
. of Large-Scale Testing Programs 



CT" 

(Jn 

LT N 

OJ 

x — 1 

O' 

Lu 



't&i 1 s l l 



Z 



; it ' 
^ f I ! ! • 



cSllfx 



-Yl ? i 4 # 

? Z - - ~ 



^5 

z < r z £2 

0,5 

✓> , , 2 , 

£ ~ ; S • - ; 

5 r . 



i. 



O* 

cO 

ft 



lO 

© 

© 



Barry J. Reins teicr 
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Large-‘sc v ale testings programs are 'commonly defined as efforts 

* <t 

to determine student achievement on' a school* district, st£te, or 

* < * * * 

national basis. Further, 1 the "recent initiation of state assessment 
programs has developed as a corollary of demands for accountability 
in public education. As educational expenditures' have risen, coupled 
increased competition for rawer available dollars, the demands * 
for educational accountability have likewise increased; so, much so, 



thaf accountability has become a preeminent concern'of educational 



decision-makers. Typically, state departments of education and" local 



school districts have initiated large-scale testing programs for “the 



purposes of providing, the necessary data for more dnformeji decision-r- 
and local 



f making and for use in judging the effectiveness of state 
schooling efforts^ My comments tjoday will address and be 

the- more commonly cited uses of state and local district 

♦ ' , . ’’ * * < 

programs. f * 



limited to * 

1 * 



assessment 



1 would like td begin with a point-by-point review’ of the 

- , , ^ ' . A* 

implications of som^- proposed uses of state assessment data. 



1. Allocating state -gran ts-in-aid to alleviate, weaknesses 
in ins true tiopgl programs . 



Use of state assessment data for this purpose assumes that 



accurate inte 



rpretations about instruc tiorxal weaknesses can be 



made from statewide t-est data. 



or 

■ERIC 



Prepared for a Symposium -on' Dissemination ind Utilization of Large-Scale 
Test Results held at* the ■ Annual Meeting of the American „EduQa_tional * 

Research Asspciat ion, ^an Francisco, California^ April 1976.'' 

Or- ' - , 
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.Actually-, although test data are a- useful' -indicator of program • 

♦ 

• * * * P 

strength and weakness, only a limited number of programs can v 

be reliably and validly measured with present instr um ents and 
resources, Allocation of state funds only to programs in which 
measurement is, possible carries the risk of diver ting, resources 

i , * 

from p^ogtams less amenable to measurement. This Could, work 
particular hardship on upper grade and^high school programs where 
diversity and specialization of offerings may ba the key* to quality. 

*r ' ' * ' 

Another problem is the need to take into account the social, >' 

v s 

economic and educational factors in the adult community thatyln 

large measure appear to determine levels nf, studefit ' achievement . 

j f, * * 

To illustrate, in the Portland, district^-wh^re regression equations 
are used to predict mean, achievement scores for schools from SES 
data, such as median family' income, percent student attendance, 
median-gi;adg completed for adults 25 years ana older, percent free 
lunches, etc., multiple correlations of .80/ to ,9j) are commonly 
found. .The strikingly high correlation pi group -inean and 

achievement data 'suggests there is little value in reporting and 

) * / | ^ 

comparing achievement scores of schools, school systems, or regions 

. , ' / • / - 

i* , / I , * 

in the absence of such, data. Thjus, communities -with especially 
low or high social "and -economic characteristics^ cannot be regarded 

.v r / v , 1 , 

as haying especially poor or outstanding^ educational programs 
simply because the children score low and high''^Nirespectively T , on 4 
achievement" tests ^ 







A third problem is /the effect^both direct and indirect, xhSc„ 

* ' / \ K 

distributing fundi on the basis of. test* score's .might have, If 



■ -.‘3 - 



erIc 



r 



4# 



low achievement weref use$ as a measure of socio-eqonomic depriva- 
tion, it would ov^lap criteria for funding programs ~such as are , 
found in Title I ^nd stajte f^nan^Hd eqtjali^at^'ou programs* Used 
as a measure of financial need, it would -also .make it financially. 

- * ' . V *' ft 1 

profitable for school systems Ito maintain we%k programs. 



- y. . 

2. Designing instructional support, programs for teachers . 



‘ \ 



State curriculum development and #in-service programs , could -b$ 
based upon statewide test information showing comparative strengths., 
and wfeakne^ses among areas of learning, on the basis of performance 

related to standards, .or on^the* basis of achievement 4 trends. , An 

- ***** ■ . ■ * 



f K t 

important caution that should, be observed, howeveV, is that state- 
wide data may not bk* applicabl-es to any one district, so suph 



planning should be in the form ‘of“ support for those districts in ' 

r N ~ t „ v 

which other </ata confirm that' a general" weakness disco'iferdd through 

V ’ •*. * 

| state assessment doe-s in fact apply to them. 



tWith reference to' the, setting of performance standards in state 



- assessment programs, .the following shoUld jbe considered. The 
y ' ' - . M ! ■ •*' ' • ./ , 

^ttachment of standairks Co criterion* or goal referenced tests', 

<tfte mainstay of stat^ assessment, is based upbn the mastery ' 

concept which is most: effectively applied* Co* 1 '' ' 



acquiring specific' information. "or 5 ;, skills for specific purposes. 



, within^ finite time . limits. Ttve-.lnastery concept does not apply 

* ^ - . - - 

- . J /■* 

• nearly a? appropriately, -if at. all, to long, term developmental 



types of learning such as are re-pscas^nted . in reading, writing, r 

* i * , - ,> i 

and math problem solving. If we attempt to set" standards on this * • 

. type o,f learning, they will.-a’Imost invariably create pressures. , - * 

' 4 v * * ' * , * 

for some students and be too easily -achieved by Others. 

> t . .<• , T 
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Since the designation of arbitrary standards for total t\s t or\ 
specific goal performance has many illogical and potentially 

■r * 

damaging aspects, it is recommended that state support— service 
Allocation not be based on performance in relation to standards. 
A much sounder criterion'is longitudinal evidence of declining 
performance. \ 



One further comment should be made about inferring need for 
support services from statewide test data. To deduce that a 
downward trend in achievement in itself implies less adequate 
instruction, or that increases reflect better instruction over 
time, is unwarranted without additional data. Changes in 
character of student populations can affect achievement levels 

, » ' t 

in a state as well as in a locality. Achievement is also 
associated with general attitudes of youth.. The social protests 



,?f the'mid- 60 's and manifestations of this movement such as the 

‘ n. * . 

drug culture, appeared to have a depressing effect on student 



( f 

achievement. It is doubtful that: allocation of additional 



resources to in-service education, could have 'prevented a decline 

\ 

during this period. v , f 

' , , 1 

Developing state planning statements and priorities . v 

i •* , * 

Planning statements of a state educational agency should reflect 

^"^sbp^ort and monitdring posture rather than an instructional. 

\ , * ( 

management^ntent . If uniform goals or "Standards for local 



S T - 

districts are se^in state educational agency planning |Statements 
• x 

local deeds anc^ priorities may be set aside even ttfbugh they 

* * V * 



are- 



^ 5 - 



I 



.mofre valid indicators of local needs. ' It may be appropriate to 
sjt -statewide priorities, but monitoring of local performance 
ip such priority areas should not be of such character as to 



iji st 



rorce local resources and activity to be directed toward that 

J b 

eneral need unless there is clear evidence it is also' a need' 



^ <pf that local system. 



ilevi^ing state ^minimum standards for schools . 



Measurem^tp^ procedures required for this proposed use are 

somewhat unclear. State minimum standards for schools should 

\ 

he* concerned with whether or not local districts are achieving 
their own goals, given the assumption of local curriculum 
determination. No specific standards of achievement should 

>e included in state' minimum standards for schools , * since a 

\ t 

given lfevel of per formanc^may be excellent in one district 

1 x 

:.nd very commonplace in another, depending upon the social, 
Economic, and educational condition of the community population# 

Even if these factors could be controlled, setting this type 

' * * \ , 

of standard could "have adverse effects on educational programming, 

' _ ’ ' v < ’ 

such as the inevitable diversion of resources toward the 

* [ ■ * » ’I \ 

achievement of the standard, however difficult, or even impossible- 

f \ • 1 

-it, may be for some children# 



Reporting and making ; 



nils commonly cited 
witpoyt defining tho 

I i 

envis/ioned# One *poj; 
gramt-inWaid resourc 



recommendations to the 'Legislature, 



use of .large-scale- test results is not hainful 
types of repojts and recoitimendations j 

siftle type. recommendation, allocating 
es an the basis of test results, has already 



') 



/ . 
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\ , 
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$ 



been evaluated. Other types of reports could be purely infor- 

\ / . I- 

< mational . , )* 

t 

t \ 

■ ... y t ' 

Reports on comparative achievement among districts should be 

scrupulously avoided because of the ease with tfhich ^Such infor- 
mation can be misinterpreted arid misrepresented for political . • 
\ * * t , 

purposes. Experience in several sfcdjCe^ reveals such misuse to, 

„ N , ^ f 

be < a predictable consequence 'of ‘this type of analysis. 

* *, i 

6. Determining if students' are acquiring "survival level" skills • 

« - « * 7 

or "minimum competencies 11 . 4 

< * t> * i , * • 

\ Quite frankly, although attention to "survival skills" or. 

\minimum competencies" is approaching epidemic proportions' 

\ ^ 0 > r * * 

among state legislatures and educational agencies, X have 
serious misgivings about the movement and the purely political* 

■ response it appears to represent. It is .well, to adopt a' cautious 

1 ' * * * ( , 

if. not wary attitude toward simple dgfinitioi®' of minimum or 

t * c 

Survival competencies when the skills, knowledge, attitudes* and . 

values needed to survive a-re ^so dependent on individual differences. 
' ^ * .1 
While one can sympathize with public frustration over basic skills, 

\ 

lack of consumer Education, weaknesses in -vocational training, 

student indifference ' to rights of others, and other, concerns , 

\ ' * • 

\ it> seems more appropriate that such concern^ s,hould be addressed 
- providing, programs that respond to. individual as we 1,1 as group 

needriy in * these specific ar'aak Pf t . concern rather than ' struggling 
to define, minimum learnings that all must acquire. 



O 

ERIC, 



Determining the ex tent, t^whioft students in a state have"- 

; ; \/y ; 7 

attained the skills, knowledge, and attitudes reflected in 
the educational goals of th^tf sthtel _ 

T“ * 7 T~~ V 

Achievement tests of, the tyrpe usually found in state assessment 
tasting cover such a limited area of state policy concerns that 
their value for this putWpe needs to be placed in perspective'. 
If one examines the'.gea^ of a number of states, it will., be 
found that they vary wifiMy; in -character. Some define broad 
areas of learning; some specify personal-social qualities that 

I * - - ‘ 

education should help citizens acquire; so.me refer to prpcedures 
' \ ' 
and programs to be established; some to equity In allocating * 

resources or providing opportunities; some to •competencies that' 

; J ' f V 

students should. acquire. Even where state goals define broad 

l . / 

areas of lemming, I achievement tests of the type found in state 1 

N * \ 

l V * / * 

assessment programs provide such, limited coverage that they' - 
haye only limited usefulness in assessing * attainment of such 
goals v 

Of greater importance, 'however, is .the inherent conflict 'between 

• > ■* .. . • 

• - - v ' * • 

local curriculum determination and the assumption of a common" 
v * ’ , . 

curriculum that is necessarily embodied in statewide t&sts. ' 

• • v ! , - ' r 

‘a* • r , '* ' . . > 

t * * i * ' 

Although this lack^of congruence i^ ptobabiy within acceptable 

, ; A 

limits | in cpnvent ion-based .studies such as language and mathe- 
matics], it is a great brohlem in such important curricular areas 

I ' ' . # ■ \ " , * ~ - 

as science and sdcial studies. 



\*. 



areas 
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Uses of Local District Testing Programs 




At this, point I would like co comment dn some uses of large- 

scale' testing in school, districts and some conditions that, should be 

' ", \ 

’ met if such uses are to be realized.. It should be observed that 

\ . . 

since local school districts have the delegated authority tp estab- 

\ _ \ o r - . 

lish, specific curricula, valid measurement of outcomes is pt least- 

\ \ 

theoretically., possibly, and thj.s "eliminates one of the major problems 
faced by state testing plb^grams , l.e.,the inability to collect data 

that adequately represents ,^he ^urriculum of any particular school ' * 

... > * 
district. Also, since local districts are the basic unit of educa-" 

tional management, and evaluation is an essential function of manage- 
ment, the obligation clearly rests upon the local district to determine 
if the learning outcomes of the System are being realized. This, 



should he the- purpose of city-wide testing. Where this capability -- 



V 



exists, it will be possible to conduct evaluation of ongoing programs, 
evaluation of specially funded programs, and research and experimen- 
tation. Based upon the information- produced by .testing for these 

- t ' - > ' \ ‘ ' 

• basic purposes, management decisions can be made about program opera- 



tion and resource allocation,"" and . the public can. be informed about the 



effectiveness of regular, special,, and .experimental programs. 



' V ' . * * . ' * * 

It is necessary at this juncture todiscuss some realities or » 
"conditions that must be '-addressed if'school district testing programs 
are" effectively to serve^ the. uses just described. While-^he heed - 
, fbr local district information about goal attainment is. ^elT-evident,' 

N 4 \ ^ 

school districts generally" provide iot it only in those areas where' 

♦ * . 

testing has traditionally heeij used': 'elementary reading, - language. . . 



A r> 



\ v 



,) ‘ - 



- N • 



I , « » > 

and mathematics , -and a smattering of.covera^ of* other subjects at 
the elementally and secondary level; This ha^ primarily been due to' 

• * > r« I ^ ' - 

several major obstacles inherent in developing measurement in. other 
a^eas of^learning^ I “should like t ;o comment briefly on these 

obstacles, for in a way it is unprofitable to 3 peak,of uses of tests 

* , ; 

when so many of those ^xi^es cannot b£ Realized becausfe ‘of the inability 

« * * , ■* *»'* . \ n \ 

of school systems and test publishers to produce the Vests necessary' 1 

x -v* , % 

• ’ . <■ , f • \ ‘ 

to ^provide* total curriculum coverage. * . \ ^ 

'* •*- ‘ * * S C i \ 

* . ’ 'I * ; "i k * > \ 

• * / ; . 

* A* first obstacle Is the difficulty of producing whil-def iped , L - 

/ » , ‘ ~ T ' * 

" . ' V 

outcome, statements in local districts. Efforts I. have observed to 
Jdetine^behavioral objectives have' been particularly disappointing in 
quality and utility for instructional planning and evaluation. In' 

e • • • 4 / \ ^ x A 

the^ Portland area we have spent; ,foiir years developing clearly stated 

r » * . . '1 tv 

• • t *, * V 

learning outcomes (called .course -gdals) in twelve, ipajar areas of 

« . # ' , T> ‘ 

instruction (Art, Biological Physical Sciepces, Business Education* 

.. .. * ' v * . * / 

Health Education, ‘Home 'Economics,- Industrial Education^ Language . 

. V * , ,,t7 * Z * 

Arts, 'Mathematics, -Music, .Physical Education, Secoiid Language, and 

Social Science) . This^is a Comprehensive and car&£tilly xf^issified, set 

' . • . v. ’ ' • ; • ' . * * K . ^ 

or learning outcomes and its purpose is to* enable t.eachepg to. select- * 

4 v r > * *■ 

rather than create such statements* This certainly ( does not solve 

the' problem Vsf making the Use of goals operational *in v instructional 

: „ ' * /' , * .1 . . ' , - 

planning .and measurement, *but it is ail important first step, 

. ' * • * - # * 

4 i ^ , , « ' * 

_ • ' S " ‘ t * * 

4 -second obstacle is the diversity & f philosophies and 'instruc- 

' , ' 
tional approaches encountered among teachers and instructional 

' . ' • ' V * *V' , ^ 

specialists. This is k special problem -in the sciences aiid social 

\ i 

studies, but ^ is something of'a problem in. almost ev£ry field -of learning. 



V * 



One aspect of this* complex problem in the science area is failure to 

* * 

distinguish between processes of acquiring, organizing, and interpret*- 

„ ' . . * * ‘ f 

ing existing, scientific' information and processes t of inquiry employed 

: < 

by . scientists to discover and validate infprmation^ Another is the 
senseless argument between advocates of process and product learning. 
Another , in social studies, is the failure to acknowledge 1 that concept 
learning must be defined by the informational loading given the con- 
cepts. Still another is the -failure to acknowledge that- outcomes are 

. v > * 

« , v l ■* , 

just as clearly t needed and useful in interdisciplinary 'planning as for* 

v, 

planning within a structured field of learning. 

' ' ' I \ ■ 

I ' 

A thi-td major obstacle to extending measurement to all fields 
* / 1 ** ^ r 
in tfhich it is needed is the rigorous anil resource-consuming require- 
ment^ of local test development; yet * local test development 1 is 
essential if validity is to be achieved within the framework of local 

curriculum autonomy. < * * 

c ‘ 

\ * . . * 

A fourth obstacle is the failure of many teachers to distinguish* 

beWeen means and ends of instruction; a “problem that has been ingrained 

by traditional dependence on ttexts and other support materials. This 

deters teacher ability and willingness to define measurable outcomes 
* ‘ < 
of learning. 



This catalog of obstacles is 'in tended /to convey the seriousness 
of problems to be faced if valid and r^liable^local measurement is to 
be achieved in enough areas t “*of learning to enable testing programs to 

- * * -k v 

be seriously regarded as a tool for local, management decisions. The 

\- 

problem obviously compounds fot's.tate measurement ‘programs based on 



. i; 



x. \ 



assumptions' of common goalsf' when in ^ reality ' common goals, do not, and 
• * ‘ ' , ' . ^ • 
according, to principle's- of local control, ^ should not exist. 

/ t ' 

* - ? ' 

finally, in Oregon, the new state minimum standards call- for 

t . 

* * ^ ' 

school systems to. define go^ls for, all courses offered, and this, 

o * * *- i / ' ' \ r ' * 

I believe, is a big step toward enabling valid 'Ideal measurements to 

- * < 5>, . # H ' * 

be developed.^ But it wifi not; itself carry us past the obstacles I 

T fc 

have outlined; for almost infinite variation of course goals are 

i v ’ #% 

. still permitted within these guidelines. Answers, if they are to be 

, ' . / * < 
found, will probably lie in- devising, more effective processes- for 

making informed? and defensible judgments abqut what specific learning 

-A 1 ' / 

* f \ * * 

the course s v and programs of. school systems should be held accountable 

V * 
to produce. * ' j 



